fix(cache): keep lightweight refs after a refresh instead of retaining Episode objects - #105
Open
nickwolf wants to merge 6 commits into
Open
fix(cache): keep lightweight refs after a refresh instead of retaining Episode objects#105nickwolf wants to merge 6 commits into
nickwolf wants to merge 6 commits into
Conversation
Member
|
It honestly looks all fine to me, and I don't mind keeping the test stuff as I was planning on eventually adding in tests at some point anyways so might as well get a couple tests going. You will need to fix up the merge conflicts whenever you get a chance ^^ |
…oad per episode Episodes returned by a section search carry librarySectionTitle only on the MediaContainer, so the attribute is None on every item. Reading it made plexapi reload the full item over the network once per episode: 65,925 requests and ~19 minutes on a full-library refresh, all while holding the cache lock, and any one of them timing out aborted the refresh.
nickwolf
force-pushed
the
fix/bound-refresh-change-lists
branch
from
July 28, 2026 17:59
24d1ccf to
2d6545c
Compare
nickwolf
marked this pull request as ready for review
July 28, 2026 18:02
Author
|
Rebased onto main — the conflicts were just my copy of #102's commit colliding with the merged one, so dropping it resolved everything cleanly. No conflicts in the actual follow-up work, and your scheduling-days feature is untouched. Tests still green. Out of draft now. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebased onto
mainnow that #102 has landed, and out of draft. The diff here is just the follow-up work.Follow-up to #102, as promised there.
#102 stopped the refresh from building one giant list of episodes, but the
addedandupdatedlists it hands back still hold fullEpisodeobjects, so the peak just moves from the fetch to whatever holds those lists. On my library (65,926 episodes) I forced the pathological case, every episode markedupdated, and the process peaked at 129.4 MiB against a 1 GB limit. Before, it was going over 1024 MiB and getting OOM-killed. The refs cost 242 bytes each.The change is a frozen
EpisodeRefdataclass carrying just the fields the post-refresh consumers actually read, plus the scheduler and status handlers updated to use it.There's a second fix in here that I think is the more interesting one. Episodes that come back from a section search have
librarySectionTitleset only on the MediaContainer, not on each Video, so the attribute isNoneon every item. plexapi treatsNoneon a partial object as "go fetch the real value" (base.py:657,if value not in (None, []): return value), which means reading that one field reloads the entire item over the network. That was 65,925 extra metadata requests and about 19 minutes per refresh, all insideself._lock, and any one of them timing out took down the whole run.iter_episodesnow stampssection.titleonto each episode before yielding, the same thing plexapi does atvideo.py:1318. Measured 200 GETs down to 0, peak 165.9 to 129.4 MiB, runtime about 20 min down to 6m22s. The existing code pays this same cost today in the scheduler'sitem.librarySectionTitle.Happy to split that stamp out into its own PR against #102 if you'd rather keep the two separate. It stands on its own and it's the smaller of the two.
Three things you should know before merging:
I've been running this on my own setup since 16 July. Nine days in, the container's memory high-water is 180 MB against a 1 GB limit, with no restarts.
Tests: there are 9 covering
EpisodeRefanditer_episodes, using fakes rather than a live server. The repo's old suite needed a real Plex server and a hardcoded episode count, and it died as collateral in 709cc76. I didn't try to revive it. If you'd rather not carry these, say so and I'll drop them.Does this look like the right approach to you?